Search for: All records

Learning outcomes are clear and concise statements that describe what students should be able to do or know at the end of a particular course. These statements are crucial in instructional planning, curriculum development, and assessment of student progress and learning. Although there is no universal guidance on how to develop learning outcomes, Bloom’s taxonomy is one widely used framework that helps instructors develop outcomes that reflect different levels of thinking, from basic remembering to creative problem-solving. This study investigates the potential of generative AI, specifically GPT-4, in classifying course learning outcomes according to their respective cognitive levels within the revised Bloom’s taxonomy. To assess the effectiveness of GenAI, we conducted a comparative study using a dataset of 1000 annotated learning outcomes. We tested multiple prompt engineering strategies, including zero-shot, few-shot, chain-of-thought, rhetorical situation, and multiple binary questions, leveraging GPT-4. Classification performance was evaluated using accuracy, Cohen’s κ, and F1-score. The results indicate that the prompt incorporating rhetorical context and domain-specific knowledge achieved the highest classification performance, while the multiple binary question approach underperformed even compared to the zero-shot method. Furthermore, we compared the best-performing prompting strategy with a state-of-the-art classification model, BERT. Although the fine-tuned BERT model showed superior performance, prompt-based classification exhibited moderate to substantial agreement with expert annotations. Overall, this article demonstrates the potential of leveraging large language models to advance both theoretical understanding and practical application within the field of education and natural language processing. 

The explosion of AI across all facets of society has given rise to the need for AI education across domains and levels. AI literacy has become an important concept in the current technological landscape, emphasizing the need for individuals to acquire the necessary knowledge and skills to engage with AI systems. This systematic review examined 47 articles published between 2019 and 2023, focusing on recent work to capture new insights and initiatives given the burgeoning of the literature on this topic. In the initial stage, we explored the dataset to identify the themes covered by the selected papers and the target population for AI literacy efforts. We identified that the articles broadly contributed to one of the following themes: a) conceptualizing AI literacy, b) prompting AI literacy efforts, and c) developing AI literacy assessment instruments. We also found that a range of populations, from pre-K students to adults in the workforce, were targeted. In the second stage, we conducted a thorough content analysis to synthesize six key constructs of AI literacy: Recognize, Know and Understand, Use and Apply, Evaluate, Create, and Navigate Ethically. We then applied this framework to categorize a range of empirical studies and identify the prevalence of each construct across the studies. We subsequently review assessment instruments developed for AI literacy and discuss them. The findings of this systematic review are relevant for formal education and workforce preparation and advancement, empowering individuals to leverage AI and drive innovation.